Domain adaptation for neural machine translation

The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, such translation models struggle when translating text of a specific domain. A domain may consist of text on a well-defined topic, or text of unknown provenance with an identifiable vocabulary distribution, or language with some other stylometric feature. While NMT models can achieve good translation performance on domain-specific data via simple tuning on a representative training corpus, such data-centric approaches have negative side-effects. These include over-fitting, brittleness, and `catastrophic forgetting' of previous training examples. In this thesis we instead explore more robust approaches to domain adaptation for NMT. We consider the case where a system is adapted to a specified domain of interest, but may also need to accommodate new language, or domain-mismatched sentences. We explore techniques relating to data selection and curriculum, model parameter adaptation procedure, and inference procedure. We show that iterative fine-tuning can achieve strong performance over multiple related domains, and that Elastic Weight Consolidation can be used to mitigate catastrophic forgetting in NMT domain adaptation across multiple sequential domains. We develop a robust variant of Minimum Risk Training which allows more beneficial use of small, highly domain-specific tuning sets than simple cross-entropy fine-tuning, and can mitigate exposure bias resulting from domain over-fitting. We extend Bayesian Interpolation inference schemes to Neural Machine Translation, allowing adaptive weighting of NMT ensembles to translate text from an unknown domain. Finally we demonstrate the benefit of multi-domain adaptation approaches for other lines of NMT research. We show that NMT systems using multiple forms of data representation can benefit from multi-domain inference approaches. We also demonstrate a series of domain adaptation approaches to mitigating the effects of gender bias in machine translation

UCAM Biomedical Translation at WMT19: Transfer Learning Multi-domain Ensembles

The 2019 WMT Biomedical translation task involved translating Medline abstracts. We approached this using transfer learning to obtain a series of strong neural models on distinct domains, and combining them into multi-domain ensembles. We further experiment with an adaptive language-model ensemble weighting scheme. Our submission achieved the best submitted results on both directions of English-Spanish

College Students’ Perceptions of Individuals with Anorexia Nervosa: Irritation and Admiration

Background: Stigmatizing attitudes against anorexia nervosa (AN) may act as barriers to treatment. Aims: Evaluated college students’ perceptions of AN as compared to major depressive disorder (MDD). Method: One-hundred two female undergraduates read vignettes describing targets with mild or severe MDD or AN, then rated biological, vanity, and self-responsibility attributions; feelings of admiration, sympathy, and anger; and behavioral dispositions toward coercion into treatment, imitation, and social distance. Results: AN was perceived more negatively than MDD in terms of vanity attributions, self-responsibility attributions, and feelings of anger, but more positively in terms of admiration and imitation. Conclusions: This research demonstrates stigma-related mixed messages received by individuals with AN, which might be useful in improving eating disorders mental health literacy

An Operation Sequence Model for Explainable Neural Machine Translation

We propose to achieve explainable neural machine translation (NMT) by changing the output representation to explain itself. We present a novel approach to NMT which generates the target sentence by monotonically walking through the source sentence. Word reordering is modeled by operations which allow setting markers in the target sentence and move a target-side write head between those markers. In contrast to many modern neural models, our system emits explicit word alignment information which is often crucial to practical machine translation as it improves explainability. Our technique can outperform a plain text system in terms of BLEU score under the recent Transformer architecture on Japanese-English and Portuguese-English, and is within 0.5 BLEU difference on Spanish-English

Community health workers and childhood obesity: combatting health disparities

Obesity is caused by a variety of contributing factors including genetics, behavior, and environment, which contribute to weight gain in children and adults. The obesity epidemic is growing rapidly, predisposing both children and adults to preventable chronic diseases such as heart disease and type 2 diabetes. Obese children often become obese adults, further contributing to the obesity epidemic and its economic consequences including higher healthcare costs and lost productivity. The obesity epidemic also exposes significant health disparities; non-Hispanic Blacks and Hispanics represent a disproportionate number of obese adults and children in the United Community Health Workers (CHWs) are uniquely positioned to support current efforts in the prevention and treatment of childhood obesity. Studies have found CHWs to be effective at increasing healthy behaviors and reducing disparities in cancer screenings for adult minority groups. CHWs can be trained to provide a variety of health services, reducing the burden of healthcare professionals, and reducing cost of care. CHWs provide peer to peer, culturally sensitive health information in an individual’s preferred language. The proposed study is a three-year randomized controlled clinical trial with 262 participants divided equally into two groups, intervention, and control. Non-Hispanic Black and Hispanic children ages 1-5 years old will be recruited from their pediatrician’s offices in the Boston Metropolitan Statistical Area (MSA). Participants will be identified and enrolled by research assistants based on language of care and BMI (body mass index) as recorded in the electronic medical record (EMR). Both groups will receive standard of care treatment throughout the study. The intervention group will additionally receive monthly in-home CHW visits for the first one and a half years. CHWs will take quarterly BMIs and provide education materials on healthy eating and physical activity. The primary outcome is BMI and the secondary outcomes will include healthy behaviors such as average weekly servings of fresh fruits and vegetables. At the end of the study period, all guardians will be given a survey to assess their opinions on the standard of care treatment and CHW interventions. CHWs are an untapped resource in the fight against childhood obesity, reducing health disparities, and the obesity epidemic. However, more research is needed in this area and the proposed study is a step toward proving their efficacy and efficiency. In the United States, the implementation of CHWs over time could make a huge impact on public health by reducing preventable chronic diseases.

Why not be Versatile? Applications of the SGNMT Decoder for Machine Translation

SGNMT is a decoding platform for machine translation which allows paring various modern neural models of translation with different kinds of constraints and symbolic models. In this paper, we describe three use cases in which SGNMT is currently playing an active role: (1) teaching as SGNMT is being used for course work and student theses in the MPhil in Machine Learning, Speech and Language Technology at the University of Cambridge, (2) research as most of the research work of the Cambridge MT group is based on SGNMT, and (3) technology transfer as we show how SGNMT is helping to transfer research findings from the laboratory to the industry, eg. into a product of SDL plc